1. Field
The embodiments discussed herein are directed to a frame interpolation technology for newly generating a nonexistent video frame by interpolation based on any two existing video frames, and a technology using such a frame interpolation technology.
2. Description of the Related Art
A liquid crystal display (LCD) used for a thin-screen television is a hold type display which continues to perform rendering until a next video frame comes. This is different from a CRT (Cathode Ray Tube) and a plasma display that are each of impulse type display, i.e., perform rendering of a momentary image on a display.
With the hold type display as such, a phenomenon called motion judder occurs to a moving object(s) in a video due to mismatching between motion interpolation and the hold type display. The motion interpolation may be performed by the chase motion of a viewer's eyeballs, and with the hold type display, the position remains the same. Conventionally, digital terrestrial broadcasting for mobiles has a frame rate of about 15 fps. This frame rate is lower than that of the current analog broadcasting, and thus the motion does not often look smooth and natural enough for viewers.
In order to solve such problems, viewers have been provided with a video through with motion interpolation, i.e., an intermediate frame is created between video frames. FIG. 18 illustrates such a motion interpolation process. A motion vector is found in a display screen from a frame 1801 at a time t-1 and a frame 1803 at a time t, and then motion interpolation may be performed with the lapse of time ½ after the time t-1, for example. Such motion interpolation accordingly generates an interpolation frame 1802 at a time t-½ between the times t-1 and t so that the resulting video can be increased in quality.
A problem with this method for finding a motion vector, is an enormous amount of calculation is required. Especially when a target for interpolation is a video with a conspicuously large motion, i.e., a video that will be noticeably affected by frame interpolation, such a search of motion vector will not be completed within any practical time. In consideration thereof, frame interpolation selectively on the basis of scene has been proposed, e.g., examples include JP-A-11-112940 and JP-A-2000-134585. With such frame interpolation selectively on the basis of scene, target frames for interpolation are only those showing certain specific characteristics of motion, thereby implementing frame interpolation.
FIG. 19 illustrates an exemplary configuration of a frame interpolation device of such a scene-basis-selection type. A scene selection section 1901 uses as a basis a motion vector found by a motion vector search section 1902 to determine whether each scene shows certain specific characteristics, and then forwards an interpolation control signal to an interpolation image generation section 1903. This interpolation control signal is turned ON (made effective) when any scene showing certain specific characteristics is detected, and is otherwise turned OFF (made ineffective). Only when the interpolation control signal is being turned ON, the interpolation image generation section 1903 generates an interpolation frame from the motion vector.
The scene showing certain specific characteristics is exemplified by a scene being scrolled at a constant speed. With such a scene being scrolled, the motion vector in the screen is required to be aligned in orientation. However, if the screen includes any portion not moving at all like merged subtitles, the scene is not selected because the motion vector thereof has a 0-component. Also in the screen, any portion showing less activity such as sky and water surface causes a wide range of deviation to the motion vector, and thus the scene will hardly be selected. If such a scene selection is not made correctly, frame interpolation will instead cause the resulting moving images look conspicuously not smooth and natural especially at the time of change of a frame-interpolated scene to/from a not-frame-interpolated scene.
As described above, a problem with such previous frame interpolation selectively on the basis of scene is how to detect any target scenes. A method addressing the problem includes searching a motion vector with a frame image area divided into a non-moving area and a moving area. This method allows detection of a correct motion vector even from a video including any portion with less activity like sky and sea, and even if the video includes any portion not moving at all, prevents the portion from being interpolated with a deviation in the direction of scrolling.
Such previous frame interpolation selectively on the basis of scene as above often does not look effective enough with digital terrestrial broadcasting for mobiles, which is with a low frame rate of about 15 fps and with a screen being small for viewing and listening. That is, because any frame from which no scene is detected is not subjected to frame interpolation, the resulting video will look quite awkward during digital terrestrial broadcasting or others with a low frame rate.
In consideration thereof, when a target for interpolation is a video with a low frame rate, the frame interpolation function may be always activated for the purpose of improving the smoothness of motion while allowing some degree of “irregularities” in the video as a result of erroneous interpolation. A frame interpolation device in which the frame interpolation function is always activated is implemented by the configuration of FIG. 19 without including the scene selection section 1901, for example. With the device configured as such, the motion vector search section 1902 makes a search of a motion vector with a frame image area divided into a non-moving area and a moving area, and then executes a weight assigning process to the moving area in accordance with an in-screen average motion vector, thereby determining the eventual motion vector.
The problem with such always-activated frame interpolation is that, when the moving area is detected in a frame not including any specific scene, various types of movements in the video will be directed in the direction of only one average motion vector. As a result of such an erroneous interpolation process, the resulting reproduced video will not look smooth and natural.
To address the problem described above, with such always-activated flame interpolation, the weight assigning process may not be executed to the moving area in accordance with an in-screen average motion vector. If this is the case, however, when the moving area is detected in a frame including any specific scene such as scene being scrolled, any portion with less activity in the video cannot be appropriately interpolated, thereby causing deterioration of the image quality.